Joint Classifier and Kernel Design

From a machine learning perspective, the analysis of gene expression data is complicated by the extremely large feature dimensionality. The presence of a large number of irrelevant features—here genes—makes such analysis prone to due to the curse of dimensionality. To overcome this limitation, Support Vector Machines (SVM) are widely employed, since it is well known that they possess good generalization properties even in the presence of irrelevant predictor variables. Motivated by error bounds from computational learning theory, we present a Bayesian generalization of the SVM that jointly learns the optimal classifier and kernel simultaneously from the data. Theoretical and experimental results are provided to show that learning the kernel results in automatic feature selection and hence mitigates the problem of large dimensionality. I. EXTENDED ABSTRACT In the traditional pattern recognition literature, the problem of cancer diagnosis using the gene expression profile of a new tissue sample and a database of previously expression profiles and their diagnoses falls under the general class of supervised pattern recognition. Given a database of training samples from N tissues, we have a set of N expression profiles x indexed by i ∈ {1, 2, . . . , N}. Each expression profile x = [x 1 , x (i) 2 , . . . , x (i) d ] ∈ R is a d-dimensional vector representing the measured expression levels of of d genes in the tissue sample. The class membership of each database sample is known and is denoted by y. In a two-class case (e.g., the tissues are either cancerous or non-cancerous), we can assume without loss of generality that y ∈ {0, 1}. Thus, the training set D consists of N sets of expression profiles and their corresponding class membership labels: D = { 〈x, y〉 : x ∈ R, y ∈ {0, 1} }N